Conversational Agents Need to Be ‘Attentive Speakers’ to Receive Conversational Feedback from Human Interlocutors

Conversational listener feedback – small verbal/vocal signals such as ‘mhm’, ‘yeah’, ‘huh?’, head gestures, and facial expressions (Allwood et al., 1992) – is an important mechanism in dialogue that enables interlocutors to e›ciently coordinate meaning and understanding. is mechanism could also be important for the interactive coordination between humans and articial conversational agents. In previous work (Buschmeier and Kopp, 2011, 2014), we dened that agents capable of dealing with listener feedback, so called ‘attentive speaker agents’, should be able (i) to interpret their interlocutors’ listener feedback, (ii) to incrementally adapt their language production to the interlocutors’ needs, and (iii) to elicit feedback when needed. Previous research found evidence that humans are generally willing to provide listener feedback in response to feedback elicitation cues produced by articial conversational agents (Misu et al., 2011; Reidsma et al., 2011). In the present work, we investigate the inžuence that a conversational agent’s behaviour and capabilities have on its human interlocutors’ willingness to provide communicative listener feedback. We present the results of a semi-autonomous interaction study between three embodied conversational agents with dišerent feedback processing behaviours: the attentive speaker agent (AS), which pays attention to its interlocutors’ feedback, attributes listening-related mental states to them, incrementally adapts its behaviour accordingly, and elicits feedback when needed; a control agent (EA), which does not pay attention to its interlocutors’ feedback, but explicitly asks, following each presentation, whether it should continue or repeat; and a control agent (NA), which does not pay attention to its interlocutors’ feedback and does not adapt its behaviour at all (see g. 1). e study followed a between-subject design. Each participant interacted with one of the embodied conversational agents in an information presentation task. Participants were told that they cannot speak freely with the agent, but may provide multimodal listener feedback, U E A C